Migrating firewall connection state for a firewall service virtual machine

ABSTRACT

For a host that executes one or more guest virtual machines (GVMs), some embodiments provide a novel virtualization architecture for utilizing a firewall service virtual machine (SVM) on the host to check the packets sent by and/or received for the GVMs. In some embodiments, the GVMs connect to a software forwarding element (e.g., a software switch) that executes on the host to connect to each other and to other devices operating outside of the host. Instead of connecting the firewall SVM to the host&#39;s software forwarding element that connects its GVMs, the virtualization architecture of some embodiments provides an SVM interface (SVMI) through which the firewall SVM can be accessed to check the packets sent by and/or received for the GVMs.

BACKGROUND

Firewalls are typically hardware appliances that are implemented at oneor more points in a network topology in an enterprise or a datacenter.With the advent of software defined networking (SDN) and networkvirtualization (NV), traditional hardware appliances do not takeadvantage of flexibility and control that is provided by SDN and networkvirtualization. However, to date, there has been a dearth of goodsolutions for incorporating firewall services in a virtualizedenvironment.

BRIEF SUMMARY

For a host that executes one or more guest virtual machines (GVMs), someembodiments provide a novel virtualization architecture for utilizing afirewall service virtual machine (SVM) on the host to check the packetssent by and/or received for the GVMs. In some embodiments, the GVMsconnect to a software forwarding element (e.g., a software switch) thatexecutes on the host to communicate with each other and with otherdevices operating outside of the host. Instead of connecting thefirewall SVM to the host's software forwarding element, thevirtualization architecture of some embodiments provides a novel SVMinterface (SVMI) through which the firewall SVM can be accessed for thepackets sent by and/or received for the GVMs.

In some embodiments, a firewall engine that executes on the hostcommunicates with the firewall SVM through the SVMI, in order to havethe SVM check the GVM packets. The firewall engine in some embodimentsreceives a set of attributes (called tuples) for a packet that isreceived for or sent by a GVM. The firewall engine forwards this set ofattributes to the firewall SVM through the SVMI. The SVM then uses theattribute set to identify a rule that has a matching attribute set andan action. If the SVM finds such a rule, it returns to the firewallengine the rule's corresponding action, which in some embodiments is an“Allow” or “Deny.” When the firewall SVM does not find a matching rule,it returns a default action in some embodiments. In other embodiments,the firewall SVM's rule set is defined such that the firewall SVMidentifies at least one matching rule for each possible packet attributeset.

Upon receiving the returned action, the firewall engine returns theaction to the module on the host that sent the attribute set to thefirewall engine in the first place, so that this module can then performthe returned action or have the returned action performed, e.g., so thatthis module can allow or drop the packet. In some embodiments, themodule that calls the firewall engine is part of (1) a port of thesoftware forwarding element to which the packet's source or destinationGVM attaches, or (2) a virtual network interface card (VNIC) of thisGVM.

Multiple packets can have the same packet attribute sets, e.g., when thepackets are part of one flow that is associated with one communicationsession between two machines. Accordingly, in addition to supplying thereturned action to the firewall rule-check initiating module, thefirewall engine of some embodiments also stores the returned action in aconnection state data store that it can subsequently use to processfirewall checks for other packets with similar attribute sets.Specifically, the connection state data store in some embodiments storesthe actions that the firewall SVM returns for different packet attributesets. In some embodiments, the connection state data storage stores eachreturned action with an identifier (e.g., a hash value) that isgenerated from the action's corresponding attribute set. Before checkingwith the firewall SVM for a particular packet attribute set, thefirewall rule engine checks the connection state data store to determinewhether this data store has a cached action for this packet set. If not,the firewall rule engine then checks with the firewall SVM. However, ifthe connection state data store has an entry for the particular packetattribute set, the firewall engine in some embodiments may foregochecking with the firewall SVM, and instead may return the cached actionof the matching entry to the rule-check initiating module.

Even when the connection state data store has a matching entry, thefirewall SVM may configure firewall engine in some embodiments to checkwith the firewall SVM (1) to identify the action to perform for allpacket attribute sets, irrespective of whether the SVM examined thepacket attribute sets before, (2) to identify the action to perform fora packet attribute set each Nth time that the firewall engine receivesthe packet attribute set, and/or (3) to relay information about a packetattribute set that the firewall engine processes by examining itsconnection state data storage. In some embodiments, the firewall SVMconfigures the firewall engine through application program interface(API) commands of the SVMI.

Through the SVMI APIs, the firewall SVM configures the rule-checkingbehavior of the firewall engine upon receiving a packet's attribute set.In addition to the above-mentioned configured behaviors (e.g., tochecking with the firewall SVM for each packet or for each Nth packet,or to relaying information about a packet), the configured behaviors insome embodiments also include other behaviors. For instance, the SVMmight configure the firewall engine with configuration rules thatspecify how the firewall engine should check a packet that is exchangedbetween source and destination GVMs that execute on the same host.Absent special configuration, such a packet would cause the firewallengine in some embodiments to check twice with the firewall SVM, oncefor the source GVM and once for the destination GVM. However, throughthe SVMI APIs of some embodiments, the firewall SVM can configure thefirewall engine to have the firewall SVM perform a check for such apacket only once, either for the source GVM or for the destination GVM.

In some embodiments, a GVM can migrate from a first host to a secondhost in a multi-host environment. For such environments, the SVMI APIsalso allow the firewall SVM to specify the firewall engine's behavior toprepare for such a GVM migration. For instance, the SVMI APIs include aset of one or more APIs that allows a firewall SVM on the GVM's firsthost or second host to obtain the set of entries in the firewallengine's connection state data store that relate to the migrating GVM.Through this API set, the SVM on the first host can receive, update, andsupply connection state information. The firewall engine of the firsthost can then send directly or indirectly (through a VM migratorexecuting on the host) the supplied connection state information to thefirewall engine of the second host. Similarly, through this API set, theSVM on the second host can receive and possibly update connection stateinformation from the firewall engine on the second host. Accordingly,this API set relieves the firewall SVMs on the first and second hostsfrom having to have a separate mechanism to synchronize their firewallstates.

In some embodiments, the firewall engine that executes on a host is aGVM-level firewall rule engine that, in addition to its first functionas the module interacting with the firewall SVM through the SVMI,performs a second firewall function. This second function entailsenforcing a second set of firewall rules that is in addition to thefirst set of firewall rules enforced by the firewall SVM. In someembodiments, the firewall engine first effectuates the firewall SVM'srule check (e.g., by communicating with the firewall SVM and/or checkingthe connection state data store that the engine maintains for the SVM),and then performs its own rule check based on the second set of firewallrules that the engine enforces itself. In other embodiments, thefirewall engine first performs its own rule check, and then effectuatesthe firewall SVM's rule check. In still other embodiments, the orderingof these rule checks, and/or the optional performance of these rulechecks are configurable by an administrator. For instance, in someembodiments, both the firewall SVM and firewall engine rule checks areperformed for one tenant with a GVM executing on a host, but only one ofthese rule checks (e.g., either the firewall SVM rule check or thefirewall engine rule check) is performed for another tenant with anotherGVM on the host.

In some embodiments, the firewall rules for the SVM and the firewallengine are specified by different tools and/or different administrators.For instance, in some embodiments, different administrators specify thefirst SVM rule set and the second firewall engine rule set, and theseadministrators have different privileges for specifying firewall rules.Also, in some embodiments, the firewall SVM is provided by one vendor(e.g., a firewall vendor), while the firewall rule engine is provided byanother vendor. In these embodiments, the firewall engine can beaccessed to specify its rule set through a firewall engine interface(FEI). In some of these embodiments, the SVMI provides a more limitedset of controls to the firewall engine than the FEI provides, becausethe SVM firewall comes from a second different vendor and hence shouldhave less control over the firewall engine of the virtualizationenvironment that is managed by a first vendor.

While the embodiments described above provide ways to integrate andcommunicate with firewall SVMs, one of ordinary skill in the art willrealize that many embodiments of the invention are equally applicable tonon-firewall SVMs. In other words, the methodologies and architecturedescribed above are used in some embodiments to integrate andcommunicate with non-firewall SVMs.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description, the Drawings and the Claims isneeded. Moreover, the claimed subject matters are not to be limited bythe illustrative details in the Summary, Detailed Description and theDrawing.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates an example of a virtualization architecture of a hostcomputing device.

FIG. 2 illustrates an example of several logical switches that aredefined by multiple software switches that execute on multiple hosts.

FIG. 3 illustrates an example of firewall rules that are specified interms of the traditional five-tuple packet identifiers.

FIG. 4 illustrates an example of connection state data stored in aconnection state data storage of some embodiments.

FIG. 5 illustrates an example of how the service virtual machines (SVMs)are managed differently than the hypervisor firewall engines in someembodiments.

FIG. 6 illustrates the modules of the virtualization architecture of ahost that are used to configure the firewall rules of the firewall SVMand the hypervisor firewall engine.

FIG. 7 conceptually illustrates a process that the hypervisor firewallengine performs when it is called by a port to enforce its own firewallrules for a received packet.

FIGS. 8 and 9 conceptually illustrate processes that the firewall engineperforms when it is called by a port to have the SVM firewall enforceits firewall rules for a received packet.

FIG. 10 illustrates a virtualization architecture of a host for someembodiments of the invention.

FIG. 11 illustrates a general example of the migration operation of theVM migrator during a GVM's migration.

FIGS. 12 and 13 conceptually illustrate a process that the firewallengine performs in some embodiments when it is notified of a GVM'simpending migration.

FIGS. 14 and 15 illustrate two other processes that the firewall engineperforms in other embodiments when it is notified of a GVM's impendingmigration.

FIG. 16 presents a process for gathering connection state data based onthe configuration defined by the SVM.

FIG. 17 presents a data flow diagram that illustrates the firewall SVMusing the SVM interface to obtain connection state data from thefirewall engine on a second host to which a GVM migrated from a firsthost.

FIG. 18 illustrates a data flow diagram that shows the firewall enginechecking with the firewall SVM twice for a packet that is exchangedbetween source and destination GVMs that execute on the same host.

FIG. 19 illustrates a data flow diagram that shows the firewall enginechecking with the firewall SVM only at the source side for a packet thatis exchanged between source and destination GVMs that execute on thesame host.

FIG. 20 illustrates a data flow diagram that shows the firewall enginechecking with the firewall SVM only at the destination side for a packetthat is exchanged between source and destination GVMs that execute onthe same host.

FIG. 21 conceptually illustrates a process that the firewall engineperforms when it receives a packet that is exchanged between source anddestination GVMs that execute on the same host.

FIG. 22 conceptually illustrates an electronic system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

For a host that executes one or more guest virtual machines (GVMs), someembodiments provide a novel virtualization architecture for utilizing afirewall service virtual machine (SVM) on the host to check the packetssent by and/or received for the GVMs. In some embodiments, the GVMsconnect to a software forwarding element (e.g., a software switch) thatexecutes on the host to communicate with each other and with otherdevices operating outside of the host. Instead of connecting thefirewall SVM to the host's software forwarding element, thevirtualization architecture of some embodiments provides a novel SVMinterface (SVMI) through which the firewall SVM can be accessed for thepackets sent by and/or received for the GVMs. In this document, the term“packet” refers to a collection of bits in a particular format sentacross a network. One of ordinary skill in the art will recognize thatthe term packet may be used herein to refer to various formattedcollections of bits that may be sent across a network, such as Ethernetframes, TCP segments, UDP datagrams, IP packets, etc.

I. Architecture

FIG. 1 illustrates an example of such a virtualization architecture of ahost computing device (e.g., a server). Specifically, this figure showsa virtualization architecture 100 that includes an SVMI through which afirewall SVM can be accessed for packets sent by and/or received for theGVMs. As shown, this architecture includes several GVMs 105, a softwareforwarding element 110, a firewall engine 115, a firewall rule datastorage 120, a connection state data storage 125, a firewall interface127, an SVMI 130, a firewall SVM 135, a firewall rule data storage 140,and a connection state data storage 145. In some embodiments, thesoftware forwarding element 110, the firewall engine 115, the firewallrule data storage 120, the connection state data storage 125, thefirewall interface 127, the SVMI 130 operate in the kernel space of ahypervisor executing on the host, while the GVMs 105, the firewall SVM135, the firewall rule data storage 140, and the connection state datastorage 145 operate in the hypervisor's user space.

The GVMs are virtual machines executing on top of the hypervisor (notshown) that executes on the host. Examples of such machines includewebservers, application servers, database servers, etc. In some cases,all the GVMs belong to one entity, e.g., an enterprise that operates thehost. In other cases, the host executes in a multi-tenant environment(e.g., in a multi-tenant data center), and different GVMs may belong toone tenant or to multiple tenants.

As shown, each GVM 105 includes a virtual network interface card (VNIC)155 in some embodiments. Each VNIC is responsible for exchanging packetsbetween its VM and the software forwarding element. Each VNIC connectsto a particular port of the software forwarding element 110. Thesoftware forwarding element 110 also connects to a physical networkinterface card (NIC) (not shown) of the host. In some embodiments, theVNICs are software abstractions of a physical NIC (PNIC) that areimplemented by the virtualization software (e.g., by the hypervisor).

In some embodiments, the software forwarding element maintains a singleport 160 for each VNIC of each VM. The software forwarding element 110connects to a physical NIC (through a NIC driver (not shown)) to sendoutgoing packets and to receive incoming packets. In some embodiments,the software forwarding element 110 is defined to include a port 165that connects to the PNIC's driver to send and receive packets to andfrom the PNIC.

The software forwarding element 110 performs packet-processingoperations to forward packets that it receives on one of its ports toanother one of its ports. For example, in some embodiments, the softwareforwarding element tries to use data in the packet (e.g., data in thepacket header) to match a packet to flow based rules, and upon finding amatch, to perform the action specified by the matching rule (e.g., tohand the packet to one of its ports 160 or 165, which directs the packetto be supplied to a destination GVM or to the PNIC).

In some embodiments, the software forwarding element 110 is a softwareswitch, while in other embodiments it is a software router or a combinedsoftware switch/router. The software forwarding element 110 in someembodiments implements one or more logical forwarding elements (e.g.,logical switches or logical routers) with software forwarding elementsexecuting on other hosts in a multi-host environment. A logicalforwarding element in some embodiments can span multiple hosts toconnect GVMs that execute on different hosts but belong to one logicalnetwork. In other words, different logical forwarding elements can bedefined to specify different logical networks for different users, andeach logical forwarding element can be defined by multiple softwareforwarding elements on multiple hosts. Each logical forwarding elementisolates the traffic of the GVMs of one logical network from the GVMs ofanother logical network that is serviced by another logical forwardingelement. A logical forwarding element can connect GVMs executing on thesame host and/or different hosts.

FIG. 2 illustrates an example of several logical switches that aredefined by multiple software switches that execute on multiple hosts.Specifically, this figure illustrates eight GVMs (GVM 1 to GVM 8) thatexecute on two hosts 205 and 210 that include two software switches 215and 220. As shown, the two software switches 215 and 220 implement threelogical switches 225, 230, and 235 that connect three sets of GVMs forthree different entities (e.g., three different tenants). Logical switch225 connects GVMs 1 and 4 of host 205 and GVM 6 of host 210, logicalswitch 230 connects GVM 2 of host 205 and GVM 5 of host 210, and logicalswitch 235 connects GVMs 7 and 8 of host 210 and GVM 3 of host 205.

In hypervisors, software switches are sometimes referred to as virtualswitches because they operate in software and they provide the GVMs withshared access to the PNIC(s) of the host. However, in this document,software switches are referred to as physical switches because they areitems in the physical world. This terminology also differentiatessoftware switches from logical switches, which are abstractions of thetypes of connections that are provided by the software switches. Thereare various mechanisms for creating logical switches from softwareswitches. VXLAN provides one manner for creating such logical switches.The VXLAN standard is described in Mahalingam, Mallik; Dutt, Dinesh G.;et al. (2013-05-08), VXLAN: A Framework for Overlaying Virtualized Layer2 Networks over Layer 3 Networks, IETF.

Returning to FIG. 1, the ports of the software forwarding element 110 insome embodiments include one or more function calls to one or moremodules that implement special input/output (I/O) operations on incomingand outgoing packets that are received at the ports. In someembodiments, one or two of these function calls can be to the firewallengine 115, which, as further described below, performs firewalloperations on incoming and/or outgoing packets (i.e., on packets thatare received by the host for one of the GVMs or on packets that are sentby one of the GVMs). Other examples of I/O operations that areimplemented by the ports 160 include ARP broadcast suppressionoperations and DHCP broadcast suppression operations, as described inU.S. patent application Ser. No. 14/070,360, now published as UnitedStates Patent Publication number 2015/0058968. Other I/O operations canbe so implemented in some embodiments of the invention. By implementinga stack of such function calls, the ports can implement a chain of I/Ooperations on incoming and/or outgoing packets in some embodiments.Also, in some embodiments, other modules in the data path (such as theVNICs, etc.) implement the I/O function call operations (such as thefirewall function calls), instead of the ports.

As mentioned above, the firewall engine 115 can be called for incomingor outgoing packets to check whether such packets should be delivered toa VM or sent from a VM. To perform this check, the forwarding elementport 160 that calls the firewall engine supplies a set of attributes ofa packet that the port receives. In some embodiments, the set of packetattributes are packet identifiers, such as traditional five tupleidentifiers, which include the packet's source identifier, destinationidentifier, source port, destination port, and protocol (service).Before supplying these identifiers to the firewall engine, theforwarding element port extracts these identifiers from a packet that itreceives. In some embodiments, one or more of the identifier values canbe logical values that are defined for a logical network (e.g., can beIP addresses defined in a logical address space). In other embodiments,all of the identifier values are defined in the physical domains. Instill other embodiments, some of the identifier values are defined inlogical domain, while other identifier values are defined in thephysical domain.

In some embodiments, the firewall engine can perform the firewall rulecheck itself based on firewall rules that it enforces, or it can havethe firewall SVM perform this check based on firewall rules that thefirewall SVM enforces. In other words, the firewall engine 115 in someembodiments performs two functions, one as the module interacting withthe firewall SVM 135 through the SVMI 130, and the other as a firewallengine that enforces its own firewall rules. In the embodimentsdescribed below, the firewall engine performs these two functions for anincoming or outgoing GVM packet (i.e., for a packet received by theport) in response to two different calls from a port 160 to which theGVM connects. However, one of ordinary skill in the art will realizethat in other embodiments, the firewall engine 115 can perform these twofunctions in response to one call from the port 160.

The firewall engine 115 stores the firewall rules that it enforces inthe firewall rules data storage 120. To enforce these rules, thefirewall engine 115 tries to match the received packets attribute setwith corresponding attribute sets that are stored for the firewallrules. In some embodiments, each firewall rule in the data storage 120is specified in terms of (1) the same set of packet identifiers (e.g.,five-tuple identifiers) that the firewall engine receives from the port,and (2) an action that is typically specified as an “allow” to allow apacket through or a “deny” to drop the packet. An identifier in afirewall rule can be specified in terms of an individual value or awildcard value in some embodiments. In other embodiments, the identifiercan further be defined in terms of a set of individual values or anabstract container, such as a security group, a compute construct, anetwork construct, etc.

FIG. 3 illustrates an example of firewall rules 300 that are specifiedin terms of the traditional five-tuple packet identifiers. As shown inthis figure, these identifiers include the packet's source identifier,destination identifier, source port, destination port, and protocol(service). This figure shows that some identifiers are specified interms of individual values (e.g., IP address X for source identifier305), other identifiers are specified in terms of wildcard values (e.g.,destination identifier 310), and other identifiers are specified interms of abstract containers (e.g., Web Server container for sourceidentifier 315). The members of an abstract container can be specifiedthrough any number of mechanisms, e.g., through a list that specifiesthe IP addresses for the members of the abstract container.

To match a received set of packet attributes with the rules, thefirewall engine compares the received set of attributes with theassociated identifiers (e.g., five-tuple identifiers) of the firewallrules stored in the firewall rule data storage 120. Upon identifying amatching rule, the firewall engine 115 returns to the port the action(e.g., the Allow or the Deny) of the matching rule. In some embodiments,the firewall rule data storage 120 is defined in a hierarchical mannerto ensure that a packet rule check will match higher priority rulebefore matching a lower priority rule, when packet's identifiers matchmultiple rules. Also, in some embodiments, the firewall rule datastorage 120 contains a default rule that specifies a default action forany packet rule check that cannot match any other firewall rules; thisdefault rule will be a match for all possible set of packet identifiers,and ensures that the firewall rule engine will return an action for allreceived set of packet identifiers.

Multiple packets can have the same packet attribute sets, e.g., when thepackets are part of one flow that is associated with one communicationsession between two machines. Accordingly, in addition to supplying thereturned action to the port, the firewall engine of some embodimentsstores the returned action in a connection state data storage 125 thatit can subsequently use to process firewall checks for other packetswith similar attribute sets. Specifically, the connection state datastorage 125 in some embodiments stores the actions that the firewallengine 115 returns for different packet attribute sets. In someembodiments, the connection state data storage 125 stores each returnedaction with an identifier (e.g., a hash value) that is generated fromthe action's corresponding attribute set. Before checking with thefirewall rule data storage 120 for a particular packet attribute set,the firewall rule engine 115 of some embodiments checks the connectionstate data storage 125 to determine whether this storage has a cachedaction for this packet attribute set. If not, the firewall rule enginethen checks the firewall rule data storage 120. When the connectionstate data storage has an entry for the particular packet attribute set,the firewall engine returns the cached action of the matching entry tothe rule-check initiating port 160.

FIG. 4 illustrates an example of connection state data 400 stored in aconnection state data storage 125 of some embodiments. As shown, thisdata storage includes multiple connection state tables 405, one for eachGVM executing on the host of the firewall engine 115. Each table storesone or more connection state entries, with each entry corresponding to apacket attribute set checked by the firewall engine or firewall SVM, asdescribed below. Each entry is specified in terms of a hash value and anaction. The hash value is a value that is generated from the five-tupleidentifiers of the packets. The connection state entries include thehash values because these values make it easier for the firewall engineto search quickly the connection state tables. In addition to the hashvalue and action, the connection state entries in some embodimentsinclude other parameters, such as statistics regarding the connectionsthat are associated with each entry. For instance, in some embodiments,a connection state entry includes a count of the number of times thatthe firewall engine identified the entry as a matching entry for areceived packet (i.e., a count of the number of packets that havematched this entry).

As mentioned above, the forwarding element's port also calls thefirewall engine 115 of FIG. 1 when it wants the firewall SVM 135 toperform a firewall check for a packet that the port receives. In someembodiments, the firewall engine 115 first effectuates the firewallSVM's rule check (e.g., by communicating with the firewall SVM and/orchecking a connection state data storage that the engine maintains forthe SVM, as mentioned below), and then performs its own rule check basedon the second set of firewall rules that the engine enforces itself. Inother embodiments, the firewall engine first performs its own rulecheck, and then effectuates the firewall SVM's rule check. In stillother embodiments, the ordering of these rule checks, and/or theoptional performance of these rule checks are configurable by anadministrator. For instance, in some embodiments, both the firewall SVMand firewall engine rule checks are performed for one tenant with a GVMexecuting on a host, but only one of these rule checks (e.g., either thefirewall SVM rule check or the firewall engine rule check) is performedfor another tenant with another GVM on the host.

In different embodiments, the firewall engine 115 uses differentmechanisms to determine that it has to call the firewall SVM 135 toperform a firewall rule check for a received packet attribute set. Insome embodiments, the firewall engine 115 ascertains that it has to callthe firewall SVM 135 when it receives a particular type of call from theport 160. In response to such a call, it directly calls the firewall SVM135. In other embodiments, after receiving the call from the port 160,the firewall engine 115 matches the received packet attribute set to arule in the firewall rule data storage 120, or a rule in anotherfirewall rule data storage (not shown) that it maintains separately forthe firewall SVM. This rule then contains a redirect command thatdirects the firewall engine 115 to redirect the received packetattribute set to the firewall SVM 135, so that this SVM can perform therule check. Examples of both these approaches will be further describedbelow by reference to FIGS. 8 and 9.

The firewall engine communicates with the firewall SVM 135 through theSVMI 130, in order to have the SVM check the GVM's packets. For thischeck, the firewall engine in some embodiments receives the set ofpacket attributes (e.g., the five tuples) that the port identified for(e.g., extracted from) the packet. The firewall engine 115 forwards thereceived set of attributes to the firewall SVM 135 through the SVMI 130.The SVM 135 then uses the attribute set to identify a rule in itsfirewall rule data storage 140 that has a matching attribute set and anaction. If the SVM finds such a rule, it returns to the firewall enginethe rule's corresponding action, which in some embodiments is an “Allow”or “Deny.” When the firewall SVM 135 does not find a matching rule, itreturns a default action in some embodiments. In other embodiments, thefirewall SVM's rule set is defined such that the firewall SVM canidentify at least one matching rule for each possible packet attributeset. In some embodiments, the firewall data storage 140 has a structuresimilar to the structure of the firewall data storage that was describedabove by reference to FIG. 3.

As shown, the firewall SVM also has a connection state data storage 145.Whenever the firewall SVM identifies a rule in the firewall rule datastorage 140 that matches a particular received set of packet attributes,the firewall SVM not only returns the matching rules action, but alsostores the returned action in the connection state data storage 140 thatit can subsequently use to process firewall checks for other packetswith similar attribute sets. In some embodiments, the connection statedata storage stores each returned action with an identifier (e.g., ahash value) that is generated from the action's corresponding attributeset (e.g., corresponding five tuples). Before checking with the firewalldata storage 140 for a particular packet attribute set, the firewall SVM135 checks the connection state data storage 145 to determine whetherthis storage has a cached action for this attribute set. If not, thefirewall SVM 135 checks the firewall rule data storage 140. When theconnection state data storage 145 has an entry for the particular packetattribute set, the firewall SVM 135 returns the cached action of thematching entry to the firewall engine 115 through the SVMI. In someembodiments, the connection state data storage 145 has a structuresimilar to the structure of the connection state data storage that wasdescribed above by reference to FIG. 4.

Upon receiving the returned action, the firewall engine 115 returns theaction to the port 160 that sent the attribute set to the firewallengine in the first place, so that the port can perform the returnedaction or have the returned action performed, e.g., so that this modulecan allow or drop the packet. In addition to supplying the returnedaction to the port, the firewall engine of some embodiments also storesthe returned action in the connection state data storage 125 so that itcan subsequently use this record to process firewall checks for otherpackets with similar attribute sets.

Specifically, the connection state data storage 125 in some embodimentsstores the actions that the firewall SVM 135 returns for differentpacket attribute sets. As mentioned above, the connection state datastorage 125 in some embodiments stores each returned action with anidentifier (e.g., a hash value) that is generated from the action'scorresponding attribute set. Before checking with the firewall SVM for aparticular packet attribute set, the firewall rule engine checks theconnection state data storage 125 to determine whether this storage 125has a cached action for this packet set. If not, the firewall ruleengine then checks with the firewall SVM. However, if the connectionstate data storage 125 has an entry for the particular packet attributeset, the firewall engine in some embodiments may forego checking withthe firewall SVM, and instead may return the cached action of thematching entry to the rule-check initiating module. Even when theconnection state data storage 125 has a matching entry, the firewall SVM135 may configure firewall engine 115 in some embodiments to check withthe firewall SVM (1) to identify the action to perform for all packetattribute sets, irrespective of whether the SVM examined the packetattribute sets before, (2) to identify the action to perform for apacket attribute set each Nth time that the firewall engine receives thepacket attribute set, and/or (3) to relay information about a packetattribute set that the firewall engine processes by examining itsconnection state data storage.

In some embodiments, the firewall SVM configures the firewall enginethrough application program interface (API) commands of the SVMI.Through the SVMI APIs, the firewall SVM 135 configures the rule-checkingbehavior of the firewall engine 115 upon receiving a packet's attributeset. In addition to the above-mentioned configured behaviors (e.g., tochecking with the firewall SVM for each packet or for each Nth packet,or to relaying information about a packet), the configured behaviors insome embodiments also include other behaviors. For instance, the SVM 135might configure the firewall engine 115 with configuration rules thatspecify how the firewall engine should check a packet that is exchangedbetween source and destination GVMs that execute on the same host.Absent special configuration, such a packet would cause the firewallengine in some embodiments to check twice with the firewall SVM, oncefor the source GVM and once for the destination GVM. However, throughthe SVMI APIs of some embodiments, the firewall SVM can configure thefirewall engine to have the firewall SVM perform a check for such apacket only once, either for the source GVM or for the destination GVM.More details about interaction between firewall SVM and source anddestination GVMs when the source and destination GVMs are on the samehost are discussed below in Section IV.

In some embodiments, a GVM can migrate from a first host to a secondhost in a multi-host environment. For such environments, the SVMI APIsalso allow the firewall SVM to specify the firewall engine's behavior toprepare for such a GVM migration. For instance, the SVMI APIs include aset of one or more APIs that allow a firewall SVM on the GVM's firsthost or second host to obtain the set of entries in the firewallengine's connection state data storage 125 that relate to the migratingGVM. Through this API set, the SVM on the first host can receive,update, and supply connection state information. The firewall engine ofthe first host can then send directly or indirectly (through a VMmigrator executing on the host) the supplied connection stateinformation to the firewall engine of the second host. Similarly,through this API set, the SVM on the second host can receive andpossibly update connection state information (e.g., sometimes update andother times do not update) from the firewall on the second host.Accordingly, this API set relieves the firewall SVMs on the first andsecond hosts from having to have a separate mechanism to synchronizetheir firewall state. More details about migration of a GVM from onehost to another host of a datacenter and its relation with the firewallSVMs of the two hosts are discussed below in Section III.

In some embodiments, the firewall rules in the firewall rule datastorages 120 and 140 are specified by different tools and/or differentadministrators. For instance, in some embodiments, differentadministrators specify the rules in the firewall data storage 120 andthe firewall data storage 140, and these administrators have differentprivileges for specifying firewall rules. Also, in some embodiments, thefirewall SVM 135 is provided by one vendor (e.g., a firewall vendor),while the firewall rule engine 115 is provided by another vendor.

In some of these embodiments, the firewall engine can be accessed tospecify its rule set through the firewall engine interface (FEI) 115. Insome of these embodiments, the SVMI provides a more limited set ofcontrols to the firewall engine than the FEI provides, because thefirewall engine 115 is provided by the virtualizing hypervisor, whilethe SVM firewall comes from a second different vendor and hence shouldhave less control over the firewall engine of the that is managed by thehypervisor.

FIG. 5 illustrates an example of how the SVMs are managed differentlythan the hypervisor firewall engines in some embodiments. This figureillustrates multiple hosts in a datacenter. As shown, each host includesa firewall SVM 535 (like the SVM 135) and a hypervisor firewall engine515 (like firewall engine 115). As shown, the firewall SVMs 535 in someembodiments are managed by one set of SVM controllers 505, while thehypervisor firewall engines 515 are managed by another set ofcontrollers 510. Both sets of controllers communicate with the hoststhrough a network 550 (e.g., through a local area network, a wide areanetwork, a network of networks (such as the Internet), etc.). The hostsare also communicatively connected through this network 550.

The different sets of controllers might be accessible by different setsof administrators in some embodiments. For example, in the exampleillustrated in FIG. 5, the firewall engine controller set 510 can beaccessed by both tenant and datacenter administrators to specify thefirewall rules of the hypervisor firewall engines 515, while the SVMcontroller set 505 can be accessed by only tenant administrators tospecify the firewall rules of the SVM firewalls 535. In otherembodiments, the SVM controller set 505 can be accessed by onlydatacenter administrators. In still other embodiments, the firewallengine controller set 510 can be accessed by either only tenantadministrators or datacenter administrators to specify the firewallrules of the hypervisor firewall engines 515, while the SVM controllerset 505 can be accessed by both tenant and datacenter administrators tospecify the firewall rules of the SVM firewalls 535. In yet otherembodiments, both controller sets 505 and 510 can be accessed by tenantand datacenter administrators.

FIG. 6 illustrates the modules of the virtualization architecture 100 ofa host that are used to configure the firewall rules of the firewall SVMand the hypervisor firewall engine. This figure shows that the firewallengine interface 127 in some embodiments includes a firewall agent 605,a host-level firewall data storage 610, and a firewall rule publisher615. It also shows two controller interfaces 620 and 625.

The controller interface 620 is used by the firewall interface 127 tointeract with the firewall engine controller set 510 of FIG. 5 to obtainand update firewall rules for the firewall engine. In some embodiments,this interaction is managed by the firewall agent 605. This agentreceives hypervisor firewall rules from the controller set 510 throughthe controller interface 620, and stores these firewall rules in thehost-level firewall rule data storage 610. This data storage 610 storesthe firewall rules for all the GVMs executing on the host. In someembodiments, this storage also stores firewall rules from any GVM thatmay be instantiated on the host, so that in case the GVM migrates fromanother host, the rules for this GVM are already available on the host.

The publisher 615 detects changes to the firewall rules in the host datastorage 610. In response to any detected change (e.g., addition,deletion or modification of firewall rules), the publisher pushesfirewall rules that are affected by the detected change to the firewallengine 115. In some embodiments, the firewall engine maintains differentfirewall rule tables for different GVMs. In some of these embodiments,the publisher pushes the firewall rules for the different GVMs to thedifferent firewall rule tables through the firewall engine 115. When thefirewall engine is checking the firewall rules for a received packetfrom or to a GVM, the firewall engine checks the firewall rules in thefirewall rule table that it maintains for that GVM.

The controller interface 625 is used by the firewall SVM 135 to interactwith the SVM controller set 505 to obtain and update firewall rules forthe firewall SVM. The SVM receives SVM firewall rules from thecontroller set 505 through the controller interface 625, and storesthese firewall rules in the firewall rule data storage 140. In someembodiments, the SVM maintains different firewall rule tables fordifferent GVMs, while in other embodiments the SVM maintains firewallrules for different GVMs in one table.

Each controller interface 620 or 625 communicates with a controller inits corresponding controller set through a network interface (e.g., webserver) and the network 550. In the example illustrated in FIG. 6, twodifferent controller interfaces 620 and 625 are used for the firewallSVM and the hypervisor firewall engine, because these two engines areconfigured by two different controller sets. In other embodiments,however, the virtualization architecture uses one controller interfacefor both firewall engines.

The architecture 100 of FIGS. 1 and 6 was described above by referenceto numerous details. However, one of ordinary skill in the art willrealize that this architecture can be implemented differently in otherembodiments. For instance, in FIG. 1, the firewall engine 115 uses oneconnection state data storage 125 for connections (flows) checked byboth the SVM 135 and the firewall engine 115. In other embodiments,however, the firewall engine uses two different connection state datastorages, one for the connections that it checks and another one for theconnections checked by the SVM.

Other embodiments may also use different firewall rule engines. Forinstance, some embodiments use one firewall rule engine for athird-party firewall SVM and another firewall rule engine for thehypervisor. Still other embodiments use one firewall rule engine foreach tenant on a multi-tenant host. Alternatively, embodiments use thefirewall engine 115 to facilitate firewall rule checks of the SVM (i.e.,the firewall engine 115 does not perform any of its own firewall rulechecks). Also, in other embodiments, the firewall rule engine 115 is notused to access the SVM, and instead the firewall rule-checking module(e.g., the port 160) calls the SVM through the SVMI.

While the embodiments described above and below provide ways tointegrate and communicate with firewall SVMs, one of ordinary skill inthe art will realize that many embodiments of the invention are equallyapplicable to non-firewall SVMs. In other words, the methodologies andarchitecture described above are used in some embodiments to integrateand communicate with non-firewall SVMs.

II. Firewall Rule Checking Processes

FIG. 7 conceptually illustrates a process 700 that the hypervisorfirewall engine performs when it is called by a port 160 to enforce itsown firewall rules for a received packet. As shown, the process 700starts (at 705) when it receives a set of attributes to check for areceived packet. In some embodiments, the received attributes are thepacket identifiers that the port extracted from the packet. Forinstance, in some embodiments, the received attributes set includes thepacket's extracted five tuples.

Next, at 710, the process 700 determines whether it has an entry for thereceived packet attribute set in its connection state data storage 125.This is because the firewall engine may have previously performed afirewall rule check for another packet with an identical set ofattributes, and cached the result of this check in the connection statedata storage 125. As the entries in the connection state data storage125 are specified in terms of hashes of the packet attribute sets, theprocess 700 computes (at 710) a hash value from the received attributeset, and tries to match this hash value to a hash value of an entry inthe connection state data storage 125.

When the process identifies (at 710) an entry in the connection statedata storage 125 that matches the received packet attribute set (e.g.,has a hash value that matches the computed hash value), the processreturns (at 715) to the port the action of the identified entry (e.g.,the Allow or Deny), and then ends. On the other hand, when the processcannot identify (at 710) an entry in the connection state data storage125, the process identifies (at 720) the firewall rule in the firewalldata storage 120 that matches the received packet attribute set. Toidentify the firewall rule, the process 700 searches the firewall ruledata storage 120 to identify the entry that has an attribute set (e.g.,has five tuples) that match the received packets attribute set (e.g.,match the five tuples extracted from the packet).

In some embodiments, the firewall rule data storage 120 is defined in ahierarchical manner to ensure that a packet rule check will match higherpriority rule before matching a lower priority rule, when packet'sidentifiers match multiple rules. Also, in some embodiments, thefirewall rule data storage 120 contains a default rule that specifies adefault action for any packet rule check that cannot match any otherfirewall rules; this default rule will be a match for all possible setof packet identifiers, and ensures that the firewall rule engine willreturn an action for all received set of packet identifiers.

After identifying the matching firewall rule, the process 700 returns(at 725) to the port the action that is specified in the matchingfirewall rule. After 725, the process creates (730) an entry in theconnection state data storage 125 for the received attribute set and theaction that the process identified for this attribute set. To createthis entry, the process 700 in some embodiments generates a hash valuefrom the received packet attribute set, and stores this hash value alongwith the action of the matching firewall rule that it identified at 725as the entry in the connection state data storage 125. After 730, theprocess ends.

FIG. 8 conceptually illustrates a process 800 that the firewall engine115 performs when it is called by a port 160 to have the SVM firewallenforce its firewall rules for a received packet. As shown, the process800 starts (at 805) when it receives a set of attributes with a requestto have the SVM to check this received set of attributes. As mentionedabove, the received attributes are the packet identifiers (e.g., fivetuple identifiers) that the port extracted from the packet.

Next, at 810, the process 800 determines whether it has an entry for thereceived packet attribute set in the connection state data storage 125.This is because the SVM may have previously performed a firewall rulecheck for another packet with an identical set of attributes, and thefirewall engine 115 may have cached the result of this check in theconnection state data storage 125. As mentioned above, to check theconnection state data storage 125, the firewall engine 115 computes (at810) a hash value from the received attribute set, and tries to matchthis hash value to a hash value of an entry in the connection state datastorage 125.

When the process 800 identifies (at 810) an entry in the connectionstate data storage 125 that matches the received packet attribute set(e.g., identifies a hash value that matches the computed hash value),the process returns (at 815) to the port the action of the identifiedentry (e.g., the Allow or Deny). After 815, the process in someembodiments relays information to the SVM regarding the packet matchingthe cached firewall rule, and then ends.

In other embodiments, the process 800 performs differently uponidentifying an entry in the connection state data storage 125 thatmatches the received packet attribute set. For instance, in someembodiments, the process checks a configuration file (defined by thehypervisor or by the SVM) to determine whether it should first notifythe SVM of the match, before reporting the result of the match to theport. This is in case the SVM may wish to modify the returned action. Instill other embodiments, the process 800 checks with the SVM 135 evenbefore it examines (at 810) the connection state table. For instance, insome embodiments, the process 800 first checks with the SVM for aparticular attribute set, so that the SVM can first check the particularattribute set or can direct the process (i.e., the firewall engine) tocheck the connection state data storage 125 for an entry that matchesthe received attribute set. In other words, the process 800 in someembodiments checks a configuration file to determine whether it shouldcheck with the SVM 135 to see if it should examine (at 810) theconnection state data storage 125.

When the process cannot identify (at 810) an entry in the connectionstate data storage 125, the process sends (at 825) the received packetattribute set to the SVM through the SVMI. In some embodiments, the SVMperforms its firewall rule check for the received packet attribute setby examining its firewall rule data storage 140 and/or its connectionstate data storage 145, which were described above. From the SVM, theprocess 800 receives (at 830) the action of the firewall rule thatmatched the received packet attribute set.

The process 800 then returns (at 835) to the port the received action.After 835, the process creates (at 840) an entry in the connection statedata storage 125 for the received attribute set and the action receivedat 830. To create this entry, the process 800 in some embodimentsgenerates a hash value from the received packet attribute set, andstores this hash value along with the received action as the entry inthe connection state data storage 125. After 840, the process ends.

In the approach illustrated in FIG. 8, the firewall engine 115 forwardsthe firewall rule check for a received packet attribute set directly tothe firewall SVM 135 when it receives a particular type of call (e.g.,an SVM rule check call) from the port 160. However, in otherembodiments, the firewall engine 115 uses different mechanisms todetermine that it has to call the firewall SVM 135 to perform a firewallrule check for a received packet attribute set.

FIG. 9 conceptually illustrates a process 900 that employs one suchalternative mechanism. In this approach, after receiving the call fromthe port 160, the firewall engine 115 matches the received packetattribute set to a rule in the firewall rule data storage (not shown)that the firewall engine maintains separately for the firewall SVM. Thisrule then contains a redirect command that directs the firewall engine115 to redirect the received packet attribute set to the firewall SVM135, so that this SVM can perform the rule check.

The process 900 is similar to the process 800 of FIG. 8, except thatafter it determines (at 810) that a matching entry does not exist in theconnection state data storage 125, the process 9 matches (at 905) thereceived packet attribute set to a rule in the firewall rule datastorage (not shown in FIG. 1) that the firewall engine maintains for theSVM 135. This rule directs the firewall engine 115 to redirect thereceived packet attribute set to the firewall SVM 135, so that this SVMcan perform the rule check. Accordingly, after 905, the process 900sends (at 910) the received packet attribute set to the firewall SVM 135based on the redirect command of the matching firewall rule. Other thanthese two operations, the process 900 is similar to the process 800 ofFIG. 8.

III. Connection State Migration with VM Migration

A GVM in some embodiments can migrate from a first host to a second hostin a multi-host environment. For such environments, the APIs of the SVMI130 allow the firewall SVM 135 to specify the firewall engine's behaviorto prepare for such a GVM migration. For instance, the SVMI APIs includea set of one or more APIs that allow a firewall SVM on the GVM's firsthost or second host to obtain the set of entries in the firewallengine's connection state data storage 125 that relate to the migratingGVM. Through this API set, the SVM on the first host can receive,update, and supply connection state information. The firewall engine ofthe first host can then send directly or indirectly (through a VMmigrator executing on the host) the supplied connection stateinformation to the firewall engine of the second host. Similarly,through this API set, the SVM on the second host can receive andpossibly update connection state information from the firewall engine onthe second host. Accordingly, this API set relieves the firewall SVMs onthe first and second hosts from having to have a separate mechanism tosynchronize their firewall states.

FIG. 10 illustrates a virtualization architecture 1000 of a host forsome embodiments of the invention. As shown, this architecture isidentical to the architecture 100 except that it includes a VM migrator1005. The VM migrator 1005 communicates with VM migrators on other hostsin order to facilitate and coordinate the live migration of a GVM fromone host to another. Part of these coordination operations in someembodiments is the transfer of the firewall SVM connection state entriesfor the migrating VM from the connection state data storage 125 of firsthost (from which the GVM migrates) to the connection state data storage125 of the second host (to which the GVM migrates). In some embodiments,the VM migrator is part of the hypervisor virtualization application.For instance, in some embodiments, the VM migrator is the vMotionmigrator of the ESX hypervisor of VMware Inc. In some embodiments, theVM migrator operates in the kernel space of the hypervisor.

FIG. 11 illustrates a general example of the migration operation of theVM migrator 1005 during a GVM's migration. This figure shows a GVM (GVM1) migrating from a first host 1105 to a second host 1110. Specifically,in this figure, GVM 1 is shown after it has been instantiated on thesecond host 1110. Two sets of dashed lines 1120 and 1125 indicate thatGVM 1 was previously executing on the first host 1105, and that this GVMhas migrated from the first host 1105 to the second host 1110.

Another set of dashed lines 1130 conceptually represent the operation ofthe VM migrators 1005 of the hosts 1105 and 1110 in gathering theconnection state entries for GVM 1 from the connection state datastorage 125 of the first host 1105, and moving these entries to thesecond host 1110, where they are stored in the connection state datastorage 125 of the second host 1110. As shown in FIG. 11, the VMmigrators 1005 in some embodiments interact with the firewall engines125 to gather the connection state entries from the storage 125 on thehost 1105, and to supply the connection state entries to the storage 125on the host 1110. In other embodiments, the VM migrators directlyquerying the storages 125 to read and write data to these storages.

In the embodiments that store the connection state entries for a GVM inone table in the connection state data storage 125, the migrated entries1150 are all the records in the GVM's table in the connection state datastorage 125. In some embodiments, these records only include the recordsrelated to the connection state information for the firewall SVM's ruleprocessing (i.e., it includes only cached entries that are stored for anaction the firewall SVM specifies for an attribute set that itprocesses). In other embodiments, these records also include the recordsrelated to the connection state information for the firewall rules thatthe firewall engine 115 processes (i.e., it includes cached entries thatare stored for an action the firewall engine specifies for an attributeset that this engine processes).

FIG. 12 conceptually illustrates a process 1200 that the firewall engine115 performs in some embodiments when it is notified of a GVM'simpending migration. The engine 115 performs this process to inform thefirewall SVM of the impending migration, so that the SVM can obtain andpotentially update the connection state data for the migrating GVM, andthen supply the connection state data to the firewall engine, so thatthis engine can have the VM migrator 1005 move this data to the newhost. This process is described by reference to FIG. 13, which shows adata flow diagram that illustrates how the engine performs theoperations of the process 1200. The SVMI is not illustrated in FIG. 13in order to simplify this drawing. However, one or ordinary skill in theart will realize that the communication between the firewall engine 115and the SVM 135 in this figure goes through the SVMI 130.

As shown, the process 1200 initially starts (at 1205) when the firewallengine 115 receives a notification from the VM migrator 1005 that aparticular GVM is migrating from a first host (i.e., the host of thefirewall engine) to a second host. FIG. 13 illustrates this notification1305 from the VM migrator to the firewall engine.

The process then determines (at 1210) whether the migrating GVM isprotected by the firewall SVM. If not, the process 1200 notifies (at1215) the VM migrator 1005 that it does not have any SVM firewallconnection state that it has to move for the firewall SVM, and thenends. As mentioned above, in some embodiments, the firewall engineenforces its own firewall rules, and the VM migrator not only moves thefirewall SVM connection state, but also the connection state of thefirewall rules enforced by the firewall engine. The firewall engine ofthese embodiments performs a separate process to gather the connectionstate associated with its own firewall rules, and provides the gatheredstate data to the VM migrator to move the state data.

When the process determines (at 1210) that the migrating GVM isprotected by the firewall SVM, it notifies (at 1220) the firewall SVM135 of the migration of the GVM, as shown by notification 1310 from thefirewall engine to the SVM in FIG. 13. It then receives (at 1225) arequest from the firewall SVM for all connection state entries that thefirewall engine has stored in the connection state data storage 125 forthe firewall SVM processing of incoming and/or outgoing packets of themigrating GVM. This request is illustrated as communication 1315 in FIG.13.

Upon receiving the request, the process 1200 gathers (1230) theconnection state entries that the firewall engine maintains for the SVM135 and the migrating GVM in the connection state data storage 125. Thiscollection is illustrated as operations 1320 and 1325 in FIG. 13. Aftergathering this data, the process 1200 supplies (at 1230) the gatheredconnection state data to the firewall SVM, as illustrated bycommunication 1330 in FIG. 13.

The firewall SVM in some embodiments has the firewall engine gather andsupply the connection state data that it maintains for the firewall SVM,so that it can review and potentially update this data (e.g., sometimesupdate this data while other times not updating this data) before thisdata is migrated to the new host. This procedure allows the firewall SVMto capture any connection state data that is not maintained by thefirewall engine, or is not up to date in the records maintained by thefirewall engine (e.g., does not account for changes to the connectionstate that may have not yet been relayed from the SVM to the firewallengine). This procedure also relieves the firewall SVMs on differenthosts from having their own independent synchronization method, as itallows these SVMs to utilize the synchronization method of the firewallengine and the VM migrator. As mentioned above, the firewall engine andthe VM migrator are part of the hypervisor in some embodiments andperform these synchronization operations as part of the hypervisorservice functions.

After receiving the gathered connection state data, the firewall SVMreviews and potentially updates the gathered state data, and thensupplies (at 1235) the connection state data that should be migrated tothe firewall engine. The supplying of this connection state data isillustrated as operation 1335 in FIG. 13. Upon receiving the connectionstate data, the process 1200 supplies (at 1240) the received connectionstate data to the VM migrator (as shown by communication 1340 in FIG.13), so that the VM migrator 1005 can forward this state data to the VMmigrator of the second host. After 1240, the process ends.

The approach illustrated in FIGS. 12 and 13 is just one way that thefirewall SVM 135 can configure the firewall engine to provide connectionstate data for a migrating GVM. FIGS. 14 and 15 illustrate two otherapproaches. FIG. 14 illustrates an approach whereby the firewall SVM 135configures the firewall engine to gather all the connection state datafor a migrating GVM from just its own connection state data storage 125.Thus, as shown, the firewall engine 115 in this approach does notcommunicate with the firewall SVM 135 during the gathering of theconnection state data, and just gathers this data from its own localstorage 125. After gathering this data, the firewall engine suppliesthis data to the VM migrator to transfer this data to the new host ofthe migrating GVM.

In some embodiments, the firewall engine 115 uses the state gatheringapproach of FIG. 14 to gather the connection state data for thehypervisor firewall rules (the non-SVM firewall rules) that the firewallengine enforces itself. By migrating its firewall connection state data,the firewall engine allows a hypervisor firewall engine 115 on the newhost to perform its hypervisor firewall rule enforcement based on thetransferred connection state data.

FIG. 15 illustrates an approach that is the opposite approach to theapproach shown in FIG. 14 for gathering the SVM firewall connectionstate data. In this approach, the firewall SVM 135 configures thefirewall engine to gather all the connection state data for a migratingGVM form just the firewall SVM 135. Thus, as shown, the firewall engine115 in this approach does not gather any connection state data from itsown local connection data storage 125. Instead, the firewall enginecommunicates with the firewall SVM 135 to gather all of the connectionstate data for the migrating GVM from the SVM (which is provided to theSVM by the connection state data storage 145, as illustrated in thisexample). After gathering this data, the firewall engine supplies thisdata to the VM migrator to transfer this data to the new host of themigrating GVM.

In some embodiments, the firewall engine 115 performs one of theapproaches illustrated in FIGS. 13, 14 and 15 based on how it isconfigured by the firewall SVM 135 through the SVMI 130. FIG. 16presents a process 1600 that illustrates this. This process is performedby the firewall engine 115. As shown, this process initially receivesand stores (at 1605) configuration data that specifies the set ofoperations that the firewall engine has to perform to gather connectionstate data for a migrating GVM. The process receives the configurationdata from the SVM through the SVMI. In some embodiments, theconfiguration data includes a set of values (e.g., one or more flagvalues) that specify whether the firewall engine needs to collect itslocal connection state data, to supply its local connection state datato the SVM, to request connection state data from the SVM, and/or torequest review and possible update to the local connection state data bythe SVM.

After 1605, the process receives (at 1610) a notification from the VMmigrator that a GVM is migrating from the host to another host. Next, at1615, the process retrieves the stored configuration data. Based on thestored configuration data, the process then performs (at 1620) one ofthe series of operations illustrated in FIG. 13, 14 or 15 to gather theconnection state data for the migrating GVM. After 1620, the processends.

As mentioned above, a firewall SVM 135 can also use the SVMI 130 toobtain connection state data from the firewall engine 115 on a secondhost to which a GVM migrated from a first host. FIG. 17 presents a dataflow diagram that illustrates one such example. In this example, thehost is host 1110 of FIG. 11 and the migrating GVM is GVM 1. As in FIG.13, FIG. 17 does not illustrate the SVMI in order to simplify thisdrawing. However, the communication between the firewall engine 115 andthe SVM 135 in this figure is done through the SVMI 130.

As shown in FIG. 17, the data flow starts with a communication 1705 fromthe firewall engine 115 to the firewall SVM 135. This communication isabout a packet attribute set that the SVM should process, or the SVMshould know that the firewall engine processed based on the data in theconnection state data storage 125. This packet attribute set is a packetattribute set for which the connection state data storage 125 has anentry as this attribute set relates to a packet that was processed bythe SVM 135 of the first host. This entry was moved to the connectionstate data storage 125 as part of the migration of GVM 1.

In some embodiments, the communication 1705 provides an indication thatthe attribute set is an attribute set for which the connection statedata storage 125 has an entry. In other embodiments, the communication1705 does not provide such an indication, because the SVM 135 isconfigured to check with the firewall engine for connection state dataeach time that the SVM receives an attribute set that it has notpreviously processed.

When the SVM 135 gets the communication 1705, the SVM determines that itdoes not have any information regarding this packet attribute state,either from the indication in the communication 1705, or by examiningits connection state data storage 145, or through some other mechanism.In response to this determination, the SVM sends a request 1710 forconnection state information for the received attribute set to thefirewall engine 115 through the SVMI (not shown). In other words, theSVM 135 uses one of the APIs of the SVMI to ask the firewall engine 115to obtain the cached connection state data for the received attributeset from its connection state data storage 125.

The firewall engine 115 then queries 1715 and retrieves 1720 theconnection state data for the received attribute set from the connectionstate data storage, and provides this data as a reply 1725 to the APIrequest 1710. The SVM then uses this data to perform its operation,which may be to store this data with the information that it receivedwith the initial communication 1705 or to examine its firewall rules forthe attribute set that it received with the initial communication 1705.

IV. Other Configuration of Firewall Engine Behavior

Through the SVMI APIs, the firewall SVM configures the rule-checkingbehavior of the firewall engine upon receiving a packet's attribute set.For instance, as mentioned above, the firewall SVM can configure thefirewall engine to check with the firewall SVM (1) to identify theaction to perform for all packet attribute sets, irrespective of whetherthe SVM examined the packet attribute sets before, (2) to identify theaction to perform for a packet attribute set each Nth time that thefirewall engine receives the packet attribute set, and/or (3) to relayinformation about a packet attribute set that the firewall engineprocesses by examining its connection state data storage 125.

The configured behaviors in some embodiments also include otherbehaviors. For instance, the SVM might configure the firewall enginewith configuration rules that specify how the firewall engine shouldcheck a packet that is exchanged between source and destination GVMsthat execute on the same host. Absent special configuration, such apacket would cause the firewall engine in some embodiments to checktwice with the firewall SVM, once for the source GVM and once for thedestination GVM. However, through the SVMI APIs of some embodiments, thefirewall SVM can configure the firewall engine to have the firewall SVMperform a check for such a packet only once, either for the source GVMor for the destination GVM.

FIG. 18 illustrates a data flow diagram that shows the firewall engine115 checking with the firewall SVM 135 twice for a packet that isexchanged between source and destination GVMs that execute on the samehost. In this figure, the firewall engine's interaction with theconnection state data storage 125 is not show in order not to obscurethe description of this figure with unnecessary detail. As shown in FIG.18, the firewall engine 115 is called twice to process a packet sentfrom GVM 1805 to GVM 1810, a first time from the port 1815 (connected toGVM 1805) and a second time from the port 1820 (connected to GVM 1810).Each time that the firewall engine is called by a port, the engineexchanges messages with the SVM to direct the SVM to perform a firewallrule check on the packet received at the port. Having the SVM check thesame packet twice is inefficient, especially since the packet never leftthe host on which the source and destination GVMs execute.

To avoid this inefficiency, the SVM can configure the firewall enginethrough the APIs of the SVMI to have the firewall SVM only perform acheck once for a packet exchanged between source and destination GVMs onone host. This check can be performed only once for the source GVM orthe destination GVM, as shown in FIGS. 19 and 20.

FIG. 19 illustrates a data flow diagram that shows the firewall engine115 checking with the firewall SVM 135 only at the source side for apacket that is exchanged between source and destination GVMs thatexecute on the same host. In this figure, the firewall engine'sinteraction with the connection state data storage 125 is not show inorder not to obscure the description of this figure with unnecessarydetail.

FIG. 19 is identical to FIG. 18 except in two regards. The firstdifference is that FIG. 19 shows that before the packet is sent by thesource GVM 1805, the SVM 135 configures the firewall engine 115 (throughthe SVMI) to only check at the source port a packet that is exchangedbetween source and destination GVMs that execute on the same host. TheSVM configures the firewall engine by providing it with a configurationvalue 1905 that specifies the source-port only check. The firewallengine stores this configuration in a configuration data storage 1950,as represented by storage command 1910 in FIG. 19.

The second difference is that FIG. 19 shows that the firewall engine 115only checks with the SVM 135 when it gets the packet's attribute setfrom the source GVM's port 1815. This check is represented by messages1915 and 1920 that are exchanged between the firewall engine 115 and theSVM 135. The fact that the firewall engine 115 does not have the SVMperform a destination-side check on the packet is also illustrated bythe firewall engine replying with an Allow message 1930 when it receivesthe packet's attribute set from the destination GVM's port 1820. Thefirewall engine provides this reply without checking with the firewallSVM.

FIG. 20 illustrates a data flow diagram that shows the firewall engine115 checking with the firewall SVM 135 only at the destination side fora packet that is exchanged between source and destination GVMs thatexecute on the same host. Again, in this figure, the firewall engine'sinteraction with the connection state data storage 125 is not show inorder not to obscure the description of this figure with unnecessarydetail.

FIG. 20 is identical to FIG. 18 except in two regards. The firstdifference is that FIG. 20 shows that before the packet is sent by thesource GVM 1805, the SVM 135 configures the firewall engine 115 (throughthe SVMI) to only check at the destination port a packet that isexchanged between source and destination GVMs that execute on the samehost. The SVM configures the firewall engine by providing it with aconfiguration value 2005 that specifies the destination-only check. Thefirewall engine stores this configuration in a configuration datastorage 1950, as represented by storage command 2010 in FIG. 20.

The second difference is that FIG. 20 shows that the firewall engine 115only checks with the SVM 135 when it gets the packet's attribute setfrom the destination GVM's port 1820. This check is represented bymessages 2015 and 2020 that are exchanged between the firewall engine115 and the SVM 135. The fact that the firewall engine 115 does not havethe SVM perform a source-side check on the packet is also illustrated bythe firewall engine replying with an Allow message 2030 when it receivesthe packet's attribute set from the source GVM's port 1815. The firewallengine provides this reply without checking with the firewall SVM.

The examples illustrated in FIGS. 19 and 20 show that the SVM'sconfiguration of the firewall engine eliminating one redundant instanceof the firewall engine checking with the SVM when a packet is exchangedbetween source and destination GVMs executing on the same host. Evenwhen the firewall engine checks its connection state data storage 125for the connection state of packet attribute sets that it has previouslyanalyzed, the SVM's configuration of the firewall engine can alsoeliminate one of redundant checks of the connection state data storage125, by allowing this engine to only check the data storage for thesource-side or the destination-side GVM.

FIG. 21 conceptually illustrates a process 2100 that the firewall engine115 performs when it receives a packet that is exchanged between sourceand destination GVMs that execute on the same host. This process startswhen the firewall engine receives (at 2105) a packet attribute set fromeither the source or destination GVM's port. Next, at 2110, the processdetermines that the source and destination GVMs of the receivedattribute set's packet are executing on the firewall engine's host.

At 2115, the process then determines whether it has been configured toperform firewall rule processing for such a packet on behalf of the portthat provided the packet attribute set received at 2105. To do this, theprocess checks the configuration data storage 1950. If the process hasnot been configured to perform the firewall rule processing, the processreturns (at 2130) an Allow to the port that provided the received packetattribute set and then ends.

On the other hand, when the process determines (at 2115) that is hasbeen configured to perform firewall rule processing for packet attributesets received from the port that sent the current packet attribute setfor packets that are exchanged between source and destination GVMs onthe same host, the process transitions from 2115 to 2120. At 2120, theprocess performs firewall rule processing for the received packetattribute set. In some embodiments, the processing of the firewall rulesentails always forwarding any received packet attribute set to thefirewall SVM. In other embodiments, the firewall engine can process thereceived packet attribute set based on the data in its connection statedata storage 125, for some or all of the packet attributes sets that thefirewall SVM 135 has previously processed in at least one previousiteration.

After processing the firewall rules at 2120, the process determineswhether it should return a Drop or an Allow based on the matchingfirewall rule or matching connection state entry. When the matchingrecord specifies a Drop, the process transitions to 2135 to return aDrop and then ends. Alternatively, when the matching record specifies anAllow, the process transitions to 2130 to return an Allow, and thenends.

V. Electronic System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or moreprocessing unit(s) (e.g., one or more processors, cores of processors,or other processing units), they cause the processing unit(s) to performthe actions indicated in the instructions. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, RAM chips,hard drives, EPROMs, etc. The computer readable media does not includecarrier waves and electronic signals passing wirelessly or over wiredconnections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

FIG. 22 conceptually illustrates an electronic system 2200 with whichsome embodiments of the invention are implemented. The electronic system2200 can be used to execute any of the control, virtualization, oroperating system applications described above. The electronic system2200 may be a computer (e.g., a desktop computer, personal computer,tablet computer, server computer, mainframe, a blade computer etc.),phone, PDA, or any other sort of electronic device. Such an electronicsystem includes various types of computer readable media and interfacesfor various other types of computer readable media. Electronic system2200 includes a bus 2205, processing unit(s) 2210, a system memory 2225,a read-only memory 2230, a permanent storage device 2235, input devices2240, and output devices 2245.

The bus 2205 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 2200. For instance, the bus 2205 communicativelyconnects the processing unit(s) 2210 with the read-only memory 2230, thesystem memory 2225, and the permanent storage device 2235.

From these various memory units, the processing unit(s) 2210 retrieveinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Theread-only-memory (ROM) 2230 stores static data and instructions that areneeded by the processing unit(s) 2210 and other modules of theelectronic system. The permanent storage device 2235, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system2200 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 2235.

Other embodiments use a removable storage device (such as a floppy disk,flash drive, etc.) as the permanent storage device. Like the permanentstorage device 2235, the system memory 2225 is a read-and-write memorydevice. However, unlike storage device 2235, the system memory is avolatile read-and-write memory, such a random access memory. The systemmemory stores some of the instructions and data that the processor needsat runtime. In some embodiments, the invention's processes are stored inthe system memory 2225, the permanent storage device 2235, and/or theread-only memory 2230. From these various memory units, the processingunit(s) 2210 retrieve instructions to execute and data to process inorder to execute the processes of some embodiments.

The bus 2205 also connects to the input and output devices 2240 and2245. The input devices enable the user to communicate information andselect commands to the electronic system. The input devices 2240 includealphanumeric keyboards and pointing devices (also called “cursor controldevices”). The output devices 2245 display images generated by theelectronic system. The output devices include printers and displaydevices, such as cathode ray tubes (CRT) or liquid crystal displays(LCD). Some embodiments include devices such as a touchscreen thatfunction as both input and output devices.

Finally, as shown in FIG. 22, bus 2205 also couples electronic system2200 to a network 2265 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or an Intranet,or a network of networks, such as the Internet. Any or all components ofelectronic system 2200 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself.

As used in this specification, the terms “computer”, “server”,“processor”, and “memory” all refer to electronic or other technologicaldevices. These terms exclude people or groups of people. For thepurposes of the specification, the terms display or displaying meansdisplaying on an electronic device. As used in this specification, theterms “computer readable medium,” “computer readable media,” and“machine readable medium” are entirely restricted to tangible, physicalobjects that store information in a form that is readable by a computer.These terms exclude any wireless signals, wired download signals, andany other ephemeral or transitory signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. For instance, in several of theabove-described embodiments, the physical forwarding element is asoftware switch that executes on a host. In other embodiments, however,the physical forwarding element is a switch that operates in a networkinterface card (NIC) of the host.

Also, a number of the figures (e.g., FIGS. 7-9, 12, 16, and 21)conceptually illustrate processes. The specific operations of theseprocesses may not be performed in the exact order shown and described.The specific operations may not be performed in one continuous series ofoperations, and different specific operations may be performed indifferent embodiments. Furthermore, the process could be implementedusing several sub-processes, or as part of a larger macro process.

Several of the above-described embodiments involve firewall SVMs.However, as mentioned above, some of the apparatuses and methodologiesof these embodiments are equally applicable to other SVMs that provideother services (such as load balancing, network address translations,etc.). In view of the foregoing, one of ordinary skill in the art wouldunderstand that the invention is not to be limited by the foregoingillustrative details, but rather is to be defined by the appendedclaims.

We claim:
 1. A non-transitory machine readable medium storing a programfor migrating firewall connection state data as a guest virtual machine(GVM) migrates from a first host computing device to a second hostcomputing device, the program comprising sets of instructions for:receiving a configuration data set from a firewall service virtualmachine (SVM) regarding how to migrate connection state data thatrelates to firewall rule processing of the SVM for a migrating GVM;receiving indication that the GVM is migrating from the first host tothe second host; when the configuration data set is a firstconfiguration data set, performing a first set of operations to gatherthe connection state data relating to the firewall SVM's firewall ruleprocessing for the GVM; when the configuration data set is a secondconfiguration data set, performing a second set of operations to gatherthe connection state data relating to the firewall SVM's firewall ruleprocessing for the GVM; and transferring the gathered connection statedata to the second host.
 2. The non-transitory machine readable mediumof claim 1, wherein the program further comprises sets of instructionsfor: sending, to the SVM, sets of attributes of packets for the GVMs forwhich the SVM has to process firewall rules; receiving, from the SVM,actions to perform for the packets; and storing, in a data storage,connection state data based on the received actions.
 3. Thenon-transitory machine readable medium of claim 2, wherein the set ofinstructions for performing the first set of operations comprises setsof instructions for: gathering stored connection state data from thedata storage; forwarding the gathered, stored connection state data tothe SVM for the SVM to review and possibly to update; and from the SVM,receiving connection state data to transfer to the second host.
 4. Thenon-transitory machine readable medium of claim 3, wherein theforwarding and receiving of the connection state data is through an SVMinterface (SVMI); and wherein the receiving of the configuration data isalso through the SVMI.
 5. The non-transitory machine readable medium ofclaim 2, wherein the set of instructions for performing the first set ofoperations comprises a set of instructions for gathering storedconnection state data from the data storage.
 6. The non-transitorymachine readable medium of claim 1, wherein the set of instructions forperforming the first set of operations comprises sets of instructionsfor: sending the SVM a query for connection state data that is stored bythe SVM for the migrating GVM; and from the SVM, receiving connectionstate data to transfer to the second host.
 7. The non-transitory machinereadable medium of claim 6, wherein the sending of the query and thereceiving of the connection state data is through an SVM interface(SVMI); and wherein the receiving of the configuration data is alsothrough the SVMI.
 8. A non-transitory machine readable medium storing aprogram for migrating firewall connection state data as a guest virtualmachine (GVM) migrates from a first host computing device to a secondhost computing device, the program comprising sets of instructions for:receiving configuration data from a firewall service virtual machine(SVM) regarding how to migrate connection state data that relates tofirewall rule processing of the SVM for a migrating GVM; receivingindication that the GVM is migrating from the first host to the secondhost; gathering the connection state data relating to the firewall SVM'sfirewall rule processing for the GVM according to the configuration datareceived from the firewall SVM; and transferring the gathered connectionstate data to the second host by supplying the gathered connection statedata to a first GVM migrator on the first host that coordinates with asecond GVM migrator on the second host, wherein the first GVM migrator(i) transfers the gathered connection state data to the second host aspart of the transfer of the GVM from the first host to the second host,and (ii) provides the indication of the GVM migration.
 9. Anon-transitory machine readable medium storing a firewall engine modulefor migrating firewall connection state data as a guest virtual machine(GVM) migrates from a first host computing device to a second hostcomputing device, the firewall engine comprising sets of instructionsfor: receiving configuration data from a firewall service virtualmachine (SVM) regarding how to migrate connection state data thatrelates to firewall rule processing of the SVM for a migrating GVM;receiving indication that the GVM is migrating from the first host tothe second host; gathering the connection state data relating to thefirewall SVM's firewall rule processing for the GVM according to theconfiguration data received from the firewall SVM; transferring thegathered connection state data to the second host; and wherein thefirewall engine module operates in a hypervisor's kernel space; andwherein the firewall engine module enforces another set of firewallrules that is different than the set of firewall rules enforced by theSVM.
 10. The non-transitory machine readable medium of claim 1, whereinthe program further comprises sets of instructions for sending, to theSVM, sets of attributes of packets for the GVMs for which the SVM has toprocess firewall rules; the connection state data comprises a pluralityof entries, different entries correspond to different sets of packetattributes processed by the SVM; and each entry including an actionreturned by the SVM for the entry's corresponding packet attribute setand an identifier derived from the packet attribute set.
 11. Anon-transitory machine readable medium storing a program for migratingfirewall connection state data as a guest virtual machine (GVM) migratesfrom a first host computing device to a second host computing device,the program comprising sets of instructions for: receiving configurationdata from a firewall service virtual machine (SVM) regarding how tomigrate connection state data that relates to firewall rule processingof the SVM for a migrating GVM; receiving indication that the GVM ismigrating from the first host to the second host; gathering theconnection state data relating to the firewall SVM's firewall ruleprocessing for the GVM according to the configuration data received fromthe firewall SVM; transferring the gathered connection state data to thesecond host; and sending, to the SVM, sets of attributes of packets forthe GVMs for which the SVM has to process firewall rules, wherein theconnection state data comprises a plurality of entries, differententries correspond to different sets of packet attributes processed bythe SVM, each entry includes an action returned by the SVM for theentry's corresponding packet attribute set and an identifier derivedfrom the packet attribute set, and each entry's identifier is a hashvalue derived from the entry's packet attribute set.
 12. Thenon-transitory machine readable medium of claim 1, wherein once theconnection state data has been transferred to the second host, an SVM onthe second host retrieves the connection state data for a packet'sattribute set when the second host's SVM encounters an attribute setthat it has not previously received.
 13. The non-transitory machinereadable medium of claim 1, wherein once the connection state data hasbeen transferred to the second host, an SVM on the second host retrievesthe connection state data for the migrated GVM after the second host'sSVM receives a notification that the GVM has migrated to the secondhost.
 14. The non-transitory machine readable medium of claim 1, whereinonce the connection state data has been transferred to the second host,an SVM on the second host retrieves the connection state data for apacket's attribute set when the second host's SVM receives informationabout a firewall rule check that has been resolved by reference to thetransferred connection state data.
 15. The non-transitory machinereadable medium of claim 1, wherein once the connection state data hasbeen transferred to the second host, an SVM on the second host accessesthe transferred connection state data, said transfer of the connectionstate data with the migrating GVM obviating the need for the SVMs on thefirst and second hosts from having to synchronize their respectiveconnection state data.
 16. A firewall rule processing apparatus for ahost that executes a plurality of guest virtual machines (GVMs), thefirewall rule processing apparatus comprising: a non-transitorymachine-readable medium storing: a firewall service virtual machine(SVM) for processing firewall rules; a module for sending sets ofattributes of GVM packets to the firewall SVM to process the firewallrules and for storing connection state data regarding the actions thatthe firewall SVM returns for each sent set of packet attributes; and anSVM interface (SVMI) through which the SVM and module communicate, andthrough which the SVM provides a configuration data set to the module toconfigure the module to gather connection state data for a GVM thatmigrates from the host to another host, wherein based on differentprovided configuration data sets the module gathers the connection statedata differently; and a set of processors for executing the SVM and themodule.
 17. The firewall rule processing apparatus of claim 16, whereinthe provided configuration data set configures the module to follow aparticular sequence of operations to gather firewall connection statedata for the migrating GVM.
 18. The firewall rule processing apparatusof claim 17 further comprising a configuration storage in which themodule stores configuration data set received from the firewall SVMthrough the SVMI.
 19. The firewall rule processing apparatus of claim17, wherein the sequence of operations comprises: gathering storedconnection state data; forwarding the gathered, stored connection statedata to the SVM for the SVM to review and possibly to update; and fromthe SVM, receiving connection state data to transfer to the second host.20. The firewall rule processing apparatus of claim 17, wherein thesequence of operations comprises gathering stored connection state data.21. The firewall rule processing apparatus of claim 17, wherein thesequence of operations comprises: sending the SVM a query for connectionstate data that is stored by the SVM for the migrating GVM; and from theSVM, receiving connection state data to transfer to the second host. 22.The firewall rule processing apparatus of claim 16, wherein the sets ofpacket attributes comprise source, destination, source port, anddestination port data tuples.
 23. A machine-implemented method ofmigrating firewall connection state data as a guest virtual machine(GVM) migrates from a first host computing device to a second hostcomputing device, the first and second hosts executing respectivelyfirst and second firewall service virtual machines (SVMs), each SVMperforming firewall rule processing for one or more of the GVMsexecuting on the SVM's host, the method comprising: receivingconfiguration data regarding how to migrate connection state data thatrelates to firewall rule processing of the first SVM for the migratingGVM; receiving an indication that a particular GVM is migrating from thefirst host to the second host; when the configuration data set is afirst configuration data set, performing a first set of operations togather connection state data relating to the first SVM's firewall ruleprocessing for the particular GVM; when the configuration data set is asecond configuration data set, performing a second set of operations togather the connection state data relating to the firewall SVM's firewallrule processing for the GVM; and transferring the gathered connectionstate data to the second host, wherein the transferred, gatheredconnections state data allows the second SVM to process packets from themigrated particular GVM without losing the connection state data thatthe first SVM generated for the particular GVM before the GVM'smigration.
 24. The method of claim 23, wherein the transferred, gatheredconnections state data allows the second SVM to process packets from themigrated particular GVM based on the connection state data generated forthe migrated particular GVM by the first SVM.